Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[FFI] Adopt PackedFunc on Numpy Imperative Invoke #20006

Closed
wants to merge 16 commits into from

Conversation

barry-jin
Copy link
Contributor

Description

Adopt PackedFunc based FFI on imperative invoke to reduce overhead.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

@barry-jin barry-jin requested a review from szha as a code owner March 11, 2021 01:56
@mxnet-bot
Copy link

Hey @barry-jin , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [windows-gpu, centos-cpu, unix-cpu, clang, miscellaneous, website, centos-gpu, sanity, windows-cpu, unix-gpu, edge]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@lanking520 lanking520 added the pr-work-in-progress PR is still work in progress label Mar 11, 2021
@barry-jin barry-jin changed the title [WIP][FFI] Adopt PackedFunc on Numpy Imperative Invoke [FFI] Adopt PackedFunc on Numpy Imperative Invoke Apr 1, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 1, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 1, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 1, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 2, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 2, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 5, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 7, 2021
@barry-jin
Copy link
Contributor Author

@mxnet-bot run ci [all]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [centos-cpu, miscellaneous, centos-gpu, windows-gpu, unix-gpu, sanity, clang, edge, website, windows-cpu, unix-cpu]

@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 7, 2021
@leezu
Copy link
Contributor

leezu commented Apr 7, 2021

Adopt PackedFunc based FFI on imperative invoke to reduce overhead.

Have you measured the improvement for this PR? Previously I thought you intend to convert the remaining operators to the new FFI, rendering the generic imperative invoke unneeded?

@barry-jin
Copy link
Contributor Author

Adopt PackedFunc based FFI on imperative invoke to reduce overhead.

Have you measured the improvement for this PR? Previously I thought you intend to convert the remaining operators to the new FFI, rendering the generic imperative invoke unneeded?

Yes, some operators that takes no stringified params as invocation input will benefit a bit from it.
copyto

setup = """
from mxnet import np, npx
npx.set_np()
a = np.zeros((2,2))
b = np.ones((2,2))
"""
stmt = """
c = a.copyto(b)
"""
timer = timeit.Timer(setup=setup,
                     stmt=stmt)
num_repeat = 1000
print(min(timer.repeat(repeat=10, number=num_repeat)) / num_repeat)

legacy
5.821948400000032e-05
New FFI
5.494140500000011e-05

npx.gather_nd

setup = """
from mxnet import np, npx
npx.set_np()
a = np.zeros((2,2))
b = np.ones((2,2))
"""
stmt = """
c = npx.gather_nd(a, b)
"""
timer = timeit.Timer(setup=setup,
                     stmt=stmt)
num_repeat = 1000
print(min(timer.repeat(repeat=10, number=num_repeat)) / num_repeat)

legacy
8.739984900000009e-05
New FFI
7.992273099999991e-05

@barry-jin barry-jin closed this Aug 4, 2021
@barry-jin barry-jin deleted the nd_invoke branch August 4, 2021 21:23
@barry-jin barry-jin restored the nd_invoke branch August 4, 2021 21:24
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants